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Section I: The Study of Professional Development and Teacher Change: 
Building on National, Cross-Sectional Finding with Longitudinal Data 

Excerpted From: Does Professional Development Change Teaching Practice? Results 
from a Three- Year Study. 

U.S. Department of Education, Office of the Under Secretary 


Over the past decade, a large 
body of literature has emerged 
on professional development, 
teacher learning, and teacher 
changed The research litera- 
ture contains a mix of large- and 
small-scale studies, including 
intensive case studies of class- 
room teaching, evaluations of 
programs designed to improve 
teaching and learning, and sur- 
veys of teachers about their 
preservice preparation and in- 
service professional develop- 
ment experiences.^ In addition, 
there is a considerable amount 
of literature describing “best 
practices” in professional devel- 
opment, drawing on expert expe- 
riences (e.g., Loucks-Horsley et 
al., 1998). Despite the amount of 
literature, however, relatively 
little systematic research has 
been conducted on the effects of 
professional development on im- 
proving teaching or on improving 
student outcomes. 

Although relatively little re- 
search has been conducted on 
the effects of alternative forms 
of professional development, the 
research that has been con- 
ducted, along with the experi- 
ence of expert practitioners, pro- 
vides some preliminary guid- 
ance about the characteristics 
of high-quality professional de- 
velopment. Characteristics of 
professional development that 
are identified as “high quality” 
or “effective” include a focus on 
content; in-depth, active learn- 
ing opportunities; links to high 
standards, opportunities for 
teachers to engage in leader- 
ship roles; extended duration; 
and the collective participation 
of groups of teachers from the 
same school, grade, or depart- 


ment. (See, in particular. Caret 
et al., 1999; Hiebert, 1999; 
Loucks-Horsley et al. , 1998; U.S. 
Department of Education, 1999b.) 

Although lists of character- 
istics such as these commonly 
appear in the literature on ef- 
fective professional develop- 
ment, there is little direct evi- 
dence on the extent to which 
these characteristics are re- 
lated to better teaching and in- 
creased student achievement. 


Relatively little systemic re- 
search has heen conducted on 
the effects of professional de- 
velopment on improving 
teaching or on improving stu- 
dent outcomes. 


Some studies conducted over 
the past decade suggest that 
professional development expe- 
riences that share all or most 
of these characteristics can 
have a substantial, positive in- 
fluence on teachers’ classroom 
practice and student achieve- 
ment.^ Several recent studies 
have begun to examine the rela- 
tive importance of specific di- 
mensions or characteristics of 
professional development. For 
example, a number of recent 
studies suggest that the inten- 
sity and duration of professional 
development is related to the 
degree of teacher change 
(Shields, Marsh, Adelman, 
1998; Weiss, Montgomery, 
Ridgway, and Bond 1998). Fur- 
thermore, there is some indi- 
cation that professional devel- 
opment that focuses on specific 
mathematics and science con- 
tent and the ways students 
learn such content is especially 
helpful, particularly for instruc- 


tion designed to improve stu- 
dents’ conceptual understand- 
ing (Cohen Hill, 1998; 
Fennema et al., 1996). However, 
few studies have explicitly com- 
pared the effects of different 
forms of professional develop- 
ment on teaching and learn- 
ing.'* Further, most studies of 
professional development have 
not examined its effects on a 
national scale. 

Given the need for new, sys- 
tematic research on the effec- 
tiveness of alternative strate- 
gies for professional develop- 
ment, we designed our evalua- 
tion of the Eisenhower Profes- 
sional Development Program to 
enable us to examine the rela- 
tionship between professional 
development and change in 
teaching practice in both a 
cross-sectional, national prob- 
ability sample of teachers and 
a smaller, longitudinal sample 
of teachers. The Eisenhower 
program can then be evaluated 
in terms of the frequency with 
which program funds are used 
to provide professional develop- 
ment with features found to be 
effective. The results from our 
national sample of teachers are 


We found that the six key fea- 
tures of high-quality profes- 
sional development led to in- 
creases in teachers’ self-re- 
ported knowledge and skills and 
changes in teaching practice 


described in detail in our sec- 
ond-year report, Designing Effec- 
tive Professional Development: 
Lessons from the Eisenhower Pro- 
gram (Garet et al., 1999). Below 
we summarize these results 
and explain how they serve as 
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the foundation for our longitu- 
dinal study of teachers. 

What We Know About 
Eisenhower Profes- 
sional Development and 
Teacher Outcomes: 
Lessons from Our Na- 
tional Data 

The Longitudinal Study of 
Teacher Change is designed to 
build on the national, cross-sec- 
tional data that we examined in 
detail in our second-year report. 
In Caret et al. (1999), we de- 
scribed results from our 
Teacher Activity Survey, a mail 
survey of a national probability 
sample of 1,027 teachers who 
participated in 657 Eisenhower- 
assisted activities during the 
1997-98 school year.® We used 
the survey of teachers’ profes- 
sional development activities to 
assess the effectiveness of 
Eisenhower-assisted activities, 
examine the quality of 
Eisenhower-assisted activities, 
and assess the strength of the 
relationships between features 
of the activities in which teach- 
ers participated and teachers’ 
self-reported outcomes. 

To measure the quality of 
Eisenhower-assisted activities, 
we integrated and 

operationalized the ideas in the 
literature on “best practices” in 
professional development. We 
focused on three “structural fea- 
tures,” or characteristics of the 
structure of a professional de- 
velopment activity. These 
structural features include the 
form or organization of the ac- 
tivity — that is, whether the ac- 
tivity is organized as a reform 
type, such as a study group, 
teacher network, mentoring 
relationship, committee or task 
force, internship, individual re- 
search project, or teacher re- 
search center, in contrast to a 
traditional workshop, course, or 
conference; the duration of the 


activity, including the total 
number of contact hours that 
participants are expected to 
spend in the activity, as well as 
the span of time over which the 
activity takes place; and the 
degree to which the activity 
emphasizes the collective par- 
ticipation of groups of teachers 
from the same school, depart- 
ment, or grade level, as opposed 
to the participation of individual 
teachers from many schools. 

In addition to these struc- 
tural features, we focused on 
three “core features” or charac- 
teristics of the substance of the 
professional development expe- 
rience itself: the extent to which 
the activity offers opportunities 
for active learning — that is, op- 
portunities for teachers to be- 
come actively engaged in the 
meaningful analysis of teaching 
and learning, for example, by 
reviewing student work or ob- 
taining feedback on their 
teaching; the degree to which 
the activity promotes coher- 
ence in teachers’ professional 
development, by incorporating 
experiences that are consistent 
with teachers’ goals, aligned 
with state standards and as- 
sessments, and encourage con- 
tinuing professional communi- 
cation among teachers; and the 
degree to which the activity has 
a content focus — that is, the 
degree to which the activity is fo- 
cused on improving and deepening 
teachers’ content knowledge in 
mathematics and science. 

• We found that the six key fea- 
tures of high-quality profes- 
sional development led to in- 
creases in teachers’ self-re- 
ported knowledge and skills 
and changes in teaching 
practice: three structural fea- 
tures (characteristics of the 
structure of the activity) — re- 
form type, duration, and col- 
lective participation — and 
three core features (charac- 
teristics of the substance of 
the activity) — active learning. 
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coherence, and content focus. 
Our national data allowed us to 
examine how these features of 
professional development oper- 
ate to affect teacher outcomes. 
We used a statistical tech- 
nique, ordinary least squares 
regression (OLS), to estimate a 
formal causal model, which 
showed that the structural fea- 
tures of professional develop- 
ment activities influenced the 


Professional development 
that was content-focused and 
coherent and had active learn- 
ing was more successful in im- 
proving teacher knowledge 
and eliciting changes in teach- 
ers’ classroom practices. 


core features of the activities 
and that the core features, in 
turn, influenced how successful 
the experience was in increas- 
ing teacher-reported growth in 
knowledge and skills and 
changes in teaching practice. 
For example, as Exhibit 2 shows, 
activities of longer duration 
tended to place more emphasis 
on deepening teachers’ content 
knowledge, provide more oppor- 
tunities for teachers to engage 
in active learning experiences, 
and provide activities that are 
more coherent. Similarly, ac- 
tivities with greater collective 
participation of teachers also 
tended to place more emphasis 
on content, provide more oppor- 
tunities for active learning, and 
offered more coherent profes- 
sional development than other 
activities. In turn, professional 
development that was content- 
focused and coherent and had 
active learning was more suc- 
cessful in improving teacher 
knowledge and eliciting changes 
in teachers’ classroom practices. 

The features of high-quality 
professional development iden- 
tified in our national data, while 
consistent with ideas articu- 
lated in the Eisenhower legis- 
lation, deepen and extend those 
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Exhibit 2. 

The relationship Between Features of Professional Development and Teacher Outcomes 


Sponsorship Structural Features Core Features Outcomes 



Grade Level (El, Middle, High) 
In-field Certification 
Teaching Experience 


ideas by providing details about 
what makes professional devel- 
opment effective. For example, 
the Eisenhower legislation pro- 
motes professional development 
that is linked to other reform 
efforts in a coherent, system- 
atic way. The results from our 
national data show the effective- 
ness of specific dimensions of 
coherence, such as discussing 
professional development experi- 
ences with colleagues, and par- 
ticipating in follow-up activities 
that build on previous activities. 

Further, with our national 
data from district Eisenhower 
coordinators, we found signifi- 
cant differences between dis- 
tricts in the quality of profes- 
sional development they provide. 


We found these differences both 
in the features of the activities 
provided — such as active learn- 
ing, collective participation, and 
the span of time over which the 
activities extend — and in dis- 
trict management strategies, 
including alignment with stan- 
dards and assessments, fre- 
quency of co-funded projects, 
and a commitment to continu- 
ous improvement. Generally, 
we found that larger districts are 
more likely to provide high- 
quality professional develop- 
ment than are smaller districts. 
(See Caret et al., 1999, for more 
details on these findings.) 


The Purpose and De- 
sign of the Longitudinal 
Study of Teacher Change 

Our confidence in these results 
is strong, given that the data 
are from a national probability 
sample. And although the data 
are based on teacher self-re- 
ports, we have confidence in the 
validity of the data because we 
did not ask teachers to Judge the 
characteristics of the activities 
that influenced their effective- 
ness; instead we asked teach- 
ers to describe the characteris- 
tics of the activities they expe- 
rienced, and we asked them 
whether the activities had an 
effect on their knowledge, skills. 
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and classroom practice. Then, 
through data analysis tech- 
niques (e.g., ordinary least 
squares (OLS) regression), we 
identified characteristics that 


The purpose of the Longitudi- 
nal Study of Teacher Change 
is to examine the effects of 
Eisenhower-assisted and other 
professional development on 
teaching practice in math- 
ematics and science. 


were associated with the effec- 
tiveness of the activities. Be- 
cause teachers were not asked 
to judge the quality of the pro- 
fessional development in which 
they participated, the study 
minimizes self-report bias (e.g., 
Mullens 8& Gayler, 1999; 
Mullens, 1998). In addition, the 
substantial variation in the re- 
sponses that teachers and dis- 
trict administrators provided to 
these behavioral items, as well 
as the consistency in teacher 
and district administrator re- 
sponses, provides support for the 
validity of the data. 

Although these data showed 
significant relationships be- 
tween professional development 
and changes in teaching prac- 
tice, the data are cross-sec- 
tional (i.e., they were collected 
at only one point in time). A 
stronger method of attributing 
changes in teaching practice to 
professional development expe- 
riences is to gather longitudi- 
nal data on teaching practice 
and experiences in professional 
development. The Longitudinal 
Study of Teacher Change was 
designed to build on these find- 
ings from our national, cross- 
sectional data. With longitudi- 
nal data, we can add to our 
knowledge drawn from the na- 
tional data. The longitudinal 
data enable us to document 
teaching practice before and af- 
ter a professional development 
activity and to examine the ex- 


tent to which changes in teach- 
ing practice can be attributed to 
participation in the professional 
development activity. 

The purpose of the Longitu- 
dinal Study of Teacher Change 
is to examine the effects of 
Eisenhower-assisted and other 
professional development on 
teaching practice in mathemat- 
ics and science. We do not hy- 
pothesize, and so do not test, 
direct effects of professional de- 
velopment on student achieve- 
ment; rather, we examine the 
direct effects of professional de- 
velopment on teachers’ instruc- 
tion. in the LSTC, we use de- 
tailed measures of teaching 
practice that we collected by 
surveying teachers at three 
points in time: the fall of 1997, 
the spring of 1998, and the 
spring of 1999. Although our 
study does not measure the ef- 
fects of professional develop- 
ment on student achievement 
directly, the measures of teach- 
ing practice that we use have 
been associated with gains in 
student achievement. (We dis- 
cuss the measures in more de- 
tail in Section 111 of this report.) 

The Sample of Schools 

We expected systematic differ- 
ences in results by school level, 
so we chose one elementary 
school, one middle school, and 
one high school in each of the 
10 districts we studied to allow 
the analysis of results by school 
level. Further, by design, the 
sample of 30 schools in the Lon- 
gitudinal Study of Teacher 
Change is disproportionately 
high-poverty — 17 of the sample 
schools, or 57 percent, are high- 
poverty; nationwide, 25 percent 
of schools are high-poverty (de- 
fined as 50 percent or more stu- 
dents eligible for free lunch).® 
This feature of the sample is 
useful in an evaluation of the 
Eisenhower program because 
the program targets teachers in 
high-poverty schools. 


In addition, we sought 
schools in which teachers were 
likely to participate in 
Eisenhower-assisted activities 
over the 1997-98 school year, 
the year in which we conducted 
site visits to all 30 schools. ^ We 
selected states, districts, and 
schools in the sample that had 
adopted diverse approaches to 
professional development in ad- 
dition to traditional workshops 
and conferences. If such profes- 
sional development is more ef- 
fective than traditional ap- 
proaches, then the teachers’ 
instruction in the sample 
schools might be better than 
that of the average teacher. A 
few of the 30 schools experi- 
enced achievement gains in 
and 8‘*^ grade mathematics dur- 
ing the study period (1996-99), 
some experienced a decline in 
scores, and others remained at 
the same level. (See Appendix 
A for a list of the and 8‘*^ grade 

achievement scores for 1996- 
99 for each of the 30 schools). ® 
In sum, the longitudinal 
sample was selected to maxi- 
mize the opportunity to investi- 
gate important differences in 
approaches to professional de- 
velopment using Eisenhower 
funds. The sample is not na- 
tionally representative, but nei- 
ther is it extremely unusual. It 
allows an exploratory, in-depth 
examination of the character- 
istics of professional develop- 
ment that foster teacher 
change. Our longitudinal data 
complement our earlier nation- 
ally representative data. The 
national data documented the 
frequency with which 
Eisenhower professional devel- 
opment has specific character- 
istics, and our longitudinal data 
allow us to look at the effective- 
ness of these specific charac- 
teristics over time. 

The Sample of Teachers 

We surveyed all the teachers 
who taught mathematics and 
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science in each of the 30 
schools in the sample (i.e., all 
of the elementary-school teach- 
ers, and the middle and high 
school teachers who teach 
mathematics and/or science 
classes). We focus on math- 
ematics and science teachers 
because they are the primary 
participants in Eisenhower-as- 
sisted activities. In elementary 
schools, we randomly adminis- 
tered mathematics surveys to 
half the teachers and adminis- 
tered science surveys to the 
other half. The three waves of 
the survey provide data pertain- 
ing to the 1996-97, 1997-98, 
and 1998-99 school years. 

Description of the sample. 

Four hundred thirty (430) 
teachers responded to the 1996- 
97 survey; 429 teachers re- 
sponded to the 1997-98 survey; 
and 452 teachers responded to 
the 1998-99 survey. ® (See Ap- 
pendix B for a complete discus- 
sion of the response rates.) 
Some teachers who responded 
did not teach mathematics or 
science during the 1996-97, 
1997 — 98, or 1998-99 school 
year, either because they were 
not employed as teachers in one 
or more of these years or be- 
cause they taught other sub- 
jects, and thus they are not in- 
cluded in the analyses of class- 
room teaching. In addition, we 
excluded some teachers from 
particular analyses because 
they did not complete a mini- 
mum necessary set of items on 
the survey. For most analyses, 
we rely on the sample of 287 
teachers who responded to all 
three waves of the survey. For 
some analyses (those focusing 
only on professional develop- 
ment experiences), we rely on 
a sample of 318 teachers who 
responded to at least the second 
and third waves. And for some 
analyses, we restrict the 
dataset to teachers who taught 
the same course in each of the 
three years of the study (n=207). 


The response rate for the first 
wave was 75 percent; for the 
second wave, it was 74 percent; 
and for the final wave in 1998, 
75 percent. (See Appendix B for 
more details on sample sizes 
and response rates. )^° 

The sample is 74 percent fe- 
male and 18 percent minority. 
Ninety-three percent of the 
sample are certified teachers. 
Twelve percent of mathematics 
teachers and 18 percent of sci- 
ence teachers in the sample 
are novice teachers, or teach- 
ers who have taught the sur- 
veyed subject for three or fewer 
years. (See Appendix B for a 
more complete description of the 
sampling, response rates, de- 
sign, and methodology.) 

The data in this report are 
unique in that they provide detailed 
information on teaching practice 
and professional development over 
a three-year period for all teachers 
of mathematics and science in a 
school. These data enabled us to 
analyze relationships between 
teachers’ professional develop- 
ment experiences and classroom 
practice, while controlling for 
prior differences in their class- 
room practice. 

To set the context for exam- 
ining the effects of professional 
development on instruction, in 
the next section we describe the 
professional development expe- 
rienced by teachers in our lon- 
gitudinal study. 
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Endnotes 

1 See Richardson Placier (in 
press) for a comprehensive 
review of the literature on 
teacher learning and profes- 
sional development. 

2 See, for example, Cohen 
(1990) for an intensive case 
study of change in mathemat- 
ics teaching; Carey and 
Frechtling (1997) for a pro- 
gram evaluation of exemplary 
professional activities in sci- 
ence; and U.S. Department of 
Education (1999a) for a na- 
tional survey of teachers fo- 
cused on teacher preparation 
and qualifications. 

3 See, for example, Fennema et 
al. (1996), an experimental 
study examining the effects 
of Cognitively Guided In- 
struction, an intervention in 
elementary school math- 
ematics; Wilson and Ball 
(1991), an intensive case 
study of two teachers who par- 
ticipated in the Summer 
Math program; and Cohen 
and Hill (1998), which de- 
scribes the relationship be- 
tween participation in profes- 
sional development, teaching 
practice, and student 
achievement, using survey 
data from California. See 
Kennedy (1998) for a review 
of available randomized stud- 
ies examining the effects of 
teacher professional develop- 
ment on student achieve- 
ment in mathematics and 
science. See Shields, Marsh, 
and Adelman (1998) for a re- 
cent examination of the ef- 
fects of the National Science 


Foundation (NSF) Statewide 
Systemic Initiatives (SSls) on 
classroom practice in math- 
ematics and science; and 
Weiss, Montgomery, Ridgway, 
and Bond (1998) for an exami- 
nation of the effects of the 
NSF Local Systemic Change 
(LSC) initiatives. 

4 Kennedy (1998) and Cohen 
and Hill (1998) are among the 
few examples of studies that 
compare the relative effec- 
tiveness of different forms of 
professional development. 
Both studies conclude that 
professional development fo- 
cused on the teaching and 
learning of specific mathemat- 
ics and science content is 
more effective than more gen- 
eral professional development. 

5 The mail survey of teachers rep- 
resents a response rate of 72 
percent of sampled teachers. 
Details regarding sampling de- 
sign and methodology are pro- 
vided in Garet et al., 1999. 

6 We used poverty data from the 

Common Core of Data (CCD). 

7 As part of our site visits to the 

30 case study schools, we 
conducted one-time class- 
room observations of two 
teachers in each school — 
usually one mathematics 
teacher and one science 
teacher. In conjunction with 
the observations, we con- 
ducted a brief pre-observation 
interview and a somewhat 
longer post-observation inter- 
view with each of the 60 teach- 
ers we observed. The results 
of these observations are dis- 
cussed in Garet et al., 1999. 

8 The achievement data were 
collected from existing data at 
the sites. Scores were not al- 
ways available for 4* and/or 
8‘*'grade for every year. Where 
4‘*^ and/or 8*grade scores are 
not available, we provide the 
scores for the grades closest to 
4* and 8**^ grade. 

9 The response rate of high 
school teachers was higher 


than those of elementary and 
middle school teachers, per- 
haps because principals and 
department chairs in high 
school were more involved in 
administering the survey. 

10 We compared responses from 
teachers who responded only 
to wave one, teachers who 
responded to waves two and 
three, and teachers who re- 
sponded to all three waves 
and found no significant dif- 
ferences in gender, teaching 
experience, certification, pov- 
erty, and all of our measures 
of teaching practice. The one 
significant difference we 
found was that teachers who 
responded to wave one only 
were overrepresented in 
high-poverty schools, com- 
pared with those who partici- 
pated in all three waves. 

1 1 We asked teachers about 
personal background infor- 
mation, such as gender and 
years of experience, only in the 
baseline wave of the survey. 


